Context-based Normalization of Histological Stains using Deep Convolutional Features

While human observers are able to cope with variations in color and appearance of histological stains, digital pathology algorithms commonly require a well-normalized setting to achieve peak performance, especially when a limited amount of labeled data is available. This work provides a fully automated, end-to-end learning-based setup for normalizing histological stains, which considers the texture context of the tissue. We introduce Feature Aware Normalization, which extends the framework of batch normalization in combination with gating elements from Long Short-Term Memory units for normalization among different spatial regions of interest. By incorporating a pretrained deep neural network as a feature extractor steering a pixelwise processing pipeline, we achieve excellent normalization results and ensure a consistent representation of color and texture. The evaluation comprises a comparison of color histogram deviations, structural similarity and measures the color volume obtained by the different methods.Comment: In: 3rd Workshop on Deep Learning in Medical Image Analysis (DLMIA 2017

Crowdsourcing of Histological Image Labeling and Object Delineation by Medical Students

Crowdsourcing in pathology has been performed on tasks that are assumed to be manageable by nonexperts. Demand remains high for annotations of more complex elements in digital microscopic images, such as anatomical structures. Therefore, this work investigates conditions to enable crowdsourced annotations of high-level image objects, a complex task considered to require expert knowledge. 76 medical students without specific domain knowledge who voluntarily participated in three experiments solved two relevant annotation tasks on histopathological images: (1) Labeling of images showing tissue regions, and (2) delineation of morphologically defined image objects. We focus on methods to ensure sufficient annotation quality including several tests on the required number of participants and on the correlation of participants' performance between tasks. In a set up simulating annotation of images with limited ground truth, we validated the feasibility of a confidence score using full ground truth. For this, we computed a majority vote using weighting factors based on individual assessment of contributors against scattered gold standard annotated by pathologists. In conclusion, we provide guidance for task design and quality control to enable a crowdsourced approach to obtain accurate annotations required in the era of digital pathology

Exploring the spatial dimension of estrogen and progesterone signaling: detection of nuclear labeling in lobular epithelial cells in normal mammary glands adjacent to breast cancer

Peer reviewe

Fast segmentation for texture-based cartography of whole slide images

In recent years, new optical microscopes have been developed, providing very high spatial resolution images called Whole Slide Images (WSI). The fast and accurate display of such images for visual analysis by pathologists and the conventional automated analysis remain challenging, mainly due to the image size (sometimes billions of pixels) and the need to analyze certain image features at high resolution. To propose a decision support tool to help the pathologist interpret the information contained by the WSI, we present a new approach to establish an automatic cartography of WSI in reasonable time. The method is based on an original segmentation algorithm and on a supervised multiclass classification using a textural characterization of the regions computed by the segmentation. Application to breast cancer WSI shows promising results in terms of speed and quality

Synthesizing Whole Slide Images

The increasing availability of digital whole slide images opens new perspectives for computer-assisted image analysis complementing modern histopathology, assuming we can implement reliable and efficient image analysis algorithms to extract the biologically relevant information. Both validation and supervised learning techniques typically rely on ground truths manually made by human experts. However, this task is difficult, subjective and usually not exhaustive. This is a well-known issue in the field of biomedical imaging, and a common solution is the use of artificial “phantoms”. Following this trend, we study the feasibility of synthesizing artificial histological images to create perfect ground truths. In this paper, we show that it is possible to generate a synthetic whole slide image with reasonable computing resources, and we propose a way to evaluate its quality

In-silico insights on the prognostic potential of immune cell infiltration patterns in the breast lobular epithelium

Scattered inflammatory cells are commonly observed in mammary gland tissue, most likely in response to normal cell turnover by proliferation and apoptosis, or as part of immunosurveillance. In contrast, lymphocytic lobulitis (LLO) is a recurrent inflammation pattern, characterized by lymphoid cells infiltrating lobular structures, that has been associated with increased familial breast cancer risk and immune responses to clinically manifest cancer. The mechanisms and pathogenic implications related to the inflammatory microenvironment in breast tissue are still poorly understood. Currently, the definition of inflammation is mainly descriptive, not allowing a clear distinction of LLO from physiological immunological responses and its role in oncogenesis remains unclear. To gain insights into the prognostic potential of inflammation, we developed an agent-based model of immune and epithelial cell interactions in breast lobular epithelium. Physiological parameters were calibrated from breast tissue samples of women who underwent reduction mammoplasty due to orthopedic or cosmetic reasons. The model allowed to investigate the impact of menstrual cycle length and hormone status on inflammatory responses to cell turnover in the breast tissue. Our findings suggested that the immunological context, defined by the immune cell density, functional orientation and spatial distribution, contains prognostic information previously not captured by conventional diagnostic approaches. Several studies provided conclusive evidence that a delicate balance between mammary epithelial cell proliferation and apoptosis regulates homeostasis in the healthy breast tissue 1-7. After menarche, and in the absence of pregnancy, the adult female mammary gland is subjected to cyclic fluctuations depending on hormonal stimulation 1,8. In response to such systemic hormonal changes, the breast epithelium undergoes a tightly regulated sequence of cell proliferation and apoptosis during each ovarian/menstrual cycle 1-3. The peak of epithelial cell proliferation has been reported to occur during the luteal phase, suggesting a synergistic influence of steroid hormones, such as estrogen and progesterone 2-5. In turn, the peak of apoptotic activity would be expected in response to decreasing hormone levels towards the end of the menstrual cycle 2-5. However, recent histologic findings indicate that apoptosis reaches its maximum levels in the middle of the luteal phase, although there is also a peak at about the third day of the menstrual cycle 6,7. Experimental measurements of cell turnover, i.e. programmed cell death and proliferation, demonstrated that an imbalance between the mitotic and apoptotic activity might lead to malignant transformation of epithelial cells and tumorigenic processes 9-11. Indeed, excessive cell proliferation promotes accumulation of DNA damage due to insufficient timely repair and mutations 12,13. There is also recent evidence that hormones suppress effective DNA repair and alter DNA damage response (DDR) 13-15

Stain unmixing in brightfield multiplexed immunohistochemistry

Automated image analysis of multiplexed brightfield immunohistochemistry assays is a challenging objective. One central task of the analysis is the robust identification of the different stains in the image, called stain unmixing. Stain unmixing strongly depends on the method of image acquisition. Currently available multispectral cameras enable color unmixing of single fields of view (FoV), selected by matter experts (e.g. pathologists). Beyond the individual FoV approach, there is an increasing need to process larger regions or whole histopathological sections (whole slide imaging; WSI). Rapid color deconvolution in WSI is a challenge that is only partially solved. We propose a method based on a multilayer perceptron to compute dye-specific stain layers for chromogenic red and brown labeling in WSI

Graph-based description of tertiary lymphoid organs at single-cell level

Our aim is to complement observer-dependent approaches of immune cell evaluation in microscopy images with reproducible measures for spatial composition of lymphocytic infiltrates. Analyzing such patterns of inflammation is becoming increasingly important for therapeutic decisions, for example in transplantation medicine or cancer immunology. We developed a graph-based assessment of lymphocyte clustering in full whole slide images. Based on cell coordinates detected in the full image, a Delaunay triangulation and distance criteria are used to build neighborhood graphs. The composition of nodes and edges are used for classification, e.g. using a support vector machine. We describe the variability of these infiltrates on CD3/CD20 duplex staining in renal biopsies of long-term functioning allografts, in breast cancer cases, and in lung tissue of cystic fibrosis patients. The assessment includes automated cell detection, identification of regions of interest, and classification of lymphocytic clusters according to their degree of organization. We propose a neighborhood feature which considers the occurrence of edges with a certain type in the graph to distinguish between phenotypically different immune infiltrates. Our work addresses a medical need and provides a scalable framework that can be easily adjusted to the requirements of different research questions

Deep Learning for Histopathological Image Analysis

Anatomical Pathology dates back to the 19th century when Rudolf Virchow introduced his concept of cellular pathology and when the technical improvements of light microscopy enabled widespread use of structural criteria to define diseases. Since then, the quality of optical instruments has been constantly evolving. However the central element of the diagnostic process remains the knowledge and experience of pathologists visually classifying observations according to internationally agreed guidelines (e.g., World Health Organisation (WHO) classification), and much of the pre-analytical steps of specimen preparation (e.g., fixation, embedding, sectioning, staining) is only partially automated and still requires many manual steps. Thanks to the recent advent and costeffectiveness of digital scanners, tissue histopathology slides can now be fully digitized and stored as Whole Slide Images (WSI). With the availability and analysis of a much larger set of variables combined with sophisticated imaging and analytic techniques, the traditional paradigm of pathology based on visually descriptive microscopy can be complemented and substantially improved by digital pathology, utilizing screen-based visualization of digital tissue sections and novel analysis tools potentially combining the conventional evaluation by pathologists with a computerbased diagnostic aid system. A central element of such evolving medical utilities and decision support systems will be image analysis, a field in which Deep Learning (DL) has recently made immense progress, notably the work of Lecun et al. [33] on Convolutional Neural Networks (CNNs) and especially the development of very large Artificial Neural Networks (ANNs) that are revolutionizing the field. Indeed, they have surpassed all existing image processing methods in most fields (segmentation, object detection, classification, etc.). All current methods applied to histopathological image analysis will be presented as well as the future technological issues and challenges of this discipline

Towards Histopathological Stain Invariance by Unsupervised Domain Augmentation using Generative Adversarial Networks

The application of supervised deep learning methods in digital pathology is limited due to their sensitivity to domain shift. Digital Pathology is an area prone to high variability due to many sources, including the common practice of evaluating several consecutive tissue sections stained with different staining protocols. Obtaining labels for each stain is very expensive and time consuming as it requires a high level of domain knowledge. In this article, we propose an unsupervised augmentation approach based on adversarial image-to-image translation, which facilitates the training of stain invariant supervised convolutional neural networks. By training the network on one commonly used staining modality and applying it to images that include corresponding, but differently stained, tissue structures, the presented method demonstrates significant improvements over other approaches. These benefits are illustrated in the problem of glomeruli segmentation in seven different staining modalities (PAS, Jones H&E, CD68, Sirius Red, CD34, H&E and CD3) and analysis of the learned representations demonstrate their stain invariance